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Abstract 

For each finite, irreducible Coxeter system {W, S), Lusztig has associated a set of "unipotent 
characters" Uch(M^). There is also a notion of a "Fourier transform" on the space of functions 
Uch(VF) — >• K, due to Lusztig for Weyl groups and to Broue, Lusztig, and Malle in the remaining 
cases. This paper concerns a certain Vl^-representation gw in the vector space generated by the 
involutions of W. Our main result is to show that the irreducible multiplicities of gw are 
given by the Fourier transform of a unique function e : Uch(Ty) {—1, 0, 1}, which for various 
reasons serves naturally as a heuristic definition of the Frobenius-Schur indicator on Uch(l/F). 
The formula we obtain for e extends prior work of Casselman, Kottwitz, Lusztig, and Vogan 
addressing the case in which is a Weyl group. We include in addition a succinct combinatorial 
description of the irreducible decomposition of gw derived by Kottwitz when (W, S) is classical, 
and prove that gw defines a Gelfand model if and only if {W,S) has type An, H^, or him) 
with TO odd. We show finally that a conjecture of Kottwitz connecting the decomposition of 
gw to the left cells of W holds in all non-crystallographic types, and observe that a weaker 
form of Kottwitz's conjecture holds in general. In giving these results, we carefully survey the 
construction and notable properties of the set Uch(T4^) and its attached Fourier transform. 

1 Introduction 

Each finite, irreducible Coxeter system {W,S) possesses a set of "unipotent characters" Uch(W^), 
introduced by Lusztig in [26]. When (VF, S) is crystallographic, Uch(T^) arises from Lusztig's set of 
"unipotent representations" of a corresponding finite reductive group; Lusztig's resuh that this set 
depends only on (VF, S) (and not on the root datum) was a primary motivation for the definition. By 
construction, Uch(VF) always contains as a subset the set Irr(Pl^) of complex irreducible characters 
of the Coxeter group W . However, we typically view the elements ^ G Uch(T^) not as characters 
but simply as formal objects with three defining attributes: 

• A polynomial FakeDeg(<I>) G N[a;] with nonnegative integer coefficients, called the fake degree. 

• A nonzero polynomial Deg(<I>) G M[x] with real coefficients, called the (generic) degree. 

• A root of unity Eig(<I>) G C^, called the Frohenius eigenvalue. 

It takes some care to adequately describe Uch(VF) for all finite, irreducible Coxeter systems (W^ S), 
and this description is not so well-known as that of, say, Irr(VI^). Section [2.21 below supplies these 
missing details, which for the moment we can work without. 
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This paper concerns an interesting way of making sense of the question: is there a well-defined 
notion of a Frobenius-Schur indicator for a "unipotent character" $ G Uch(M^)? Many of the results 
and definitions involved in telling this story can be found scattered throughout the literature, and 
one should thus regard this paper as about 75% expository— with the remaining 25% dedicated in 
large measure to extending various results for Weyl groups to arbitrary finite Coxteter systems. 

Recall that if G is a finite group, then the Probenius-Schur indicator of an irreducible character 
$ G Irr(G) is the number 

1, if $ is the character of a representation of G in a real vector space, 
e($) = ^ 0, if the values of $ are not ah real, (1.1) 
-1, otherwise, in which case $ is called quaternionic. 

As is well-known, e($) is also the average value of ^{g^) over g £ G. Since Uch(Vl^) consists of 
formal objects and not characters, one cannot apply this definition directly, and it is not at all 
obvious what kind of definition could serve as an appropriate substitute. 

If {W, S) is crystallographic, there is an easy way of circumventing this difficulty. In this case the 
elements of Uch(l^) are in bijection with the unipotent characters of various finite reductive groups 
having W as Weyl group. While not clear a priori, it turns out that all of the actual characters 
corresponding to a given $ G Uch(I^) have the same indicator— an observation of Lusztig ^27j which 
we state precisely as Proposition 12.31 below. There is thus a logical definition of e on Uch(iy) when 
{W, S) is crystallographic, which happens to have the following simple description: define e($) to 
be 1 if Eig($) is real and otherwise. 

Things become more interesting in the case when (W, S) is non-crystallographic. In this situa- 
tion, to describe the Frobenius-Schur indicator in a satisfactory way, we require a heuristic definition 
which is consistent with the crystallographic case but which makes sense for all types. A series of 
papers appearing in the past decade, beginning with Kottwitz [20j and Casselman [12] and pro- 
ceeding through Lusztig and Vogan |28], suggests such a definition, surprisingly, in terms of the 
irreducible multiplicities of a certain l^-representation. As we will see in a moment, this approach 
unexpectedly leads to an extension of the Frobenius-Schur indicator to the non-crystallographic 
case which which forces us to assign an indicator of —1 to some elements of Uch(PF); that is, which 
suggests the existence of quaternionic "unipotent characters." 

The W^-representation of interest is given as follows. It is interesting to note that Adin, Post- 
nikov, and Roichman studied exactly this representation in type An in their paper [1]. 

Definition 1.1. Given a finite Coxeter system (W, S) with length function i : W ^ N, let 

Invol(VF) = Q-span{a^ : w e W such that = 1} 

be a vector space with a basis indexed by the involutions in W, and define qw '■ S — >• GL(Invol(VF)) 
by the formula 

J-o^, if sw = ws and e{ws) <£{w), ^ c ^ ^ w -^u 2 . 

QW[S)aw = \ tor s G D and w £ W with w = 1. 

Ittsws, otherwise. 

The map qw extends to a representation of W (a nontrivial fact, whose derivation from results 
of Lusztig and Vogan [28l [29] will be explained in Section [27T] below) . and each conjugacy class of 
involutions in W spans a ^ly-invariant subspace in Invol(VF). Let gw,a denote this subrepresentation 
on the space spanned by the conjugacy class of the involution a £ W. 
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Notation. We write xw and xw,a (when a G W and cj^ = 1) for the characters of gw and gw,a- 

Kottwitz [20j found a formula for the multiphcities of the irreducible constituents of in 
the case that (W, S) is a Weyl group, in terms of Lusztig's "non-abelian Fourier transform" [23]. 
Kottwitz proved this formula in the classical cases, while Casselman [T2] carried out the calcu- 
lations necessary to check it in the exceptional ones. In more recent work, Lusztig and Vogan 
re-encountered qw as a specialization of a certain Hecke algebra representation, and noted a way 
of restating Kottwitz's results to involve the Frobenius-Schur indicator \28\ §6.4]. 

Our main object is to describe how this last formulation extends even to the case when (W, S) 
is non-crystallographic. To this end, we must briefly introduce the Fourier transform matrix of 
Uch(iy). For any finite, irreducible Coxeter system {W,S), this is a real symmetric matrix M, 
with rows and columns indexed by Uch(VF), which possesses the following distinguishing properties 
(among others; see \n\ Theorem 6.9] and also Section [4.41 below) : 

(PI) M transforms the vector of fake degrees of Uch(VF) to the vector of (generic) degrees, per- 
muted by a certain involution (see Theorem 14.31 below). 

(P2) M is block diagonal with respect to the division of Uch(VF) into families (see Section [2.4p . 

(P3) M fixes each of the vectors indexed by Uch(14^) whose entries are the irreducible multiplicities, 
extended by zeros, of the left cell representations of W (see Theorem 15.31 below) . 

(P4) M and the diagonal matrix of Frobenius eigenvalues of Uch(14/^) determine a representation 
of the modular group PSL2(Z) (see Theorem 14.61 below) : in particular, = 1. 

Section H] below provides, with accompanying references, a careful description of the matrix M 
attached to each finite, irreducible Coxeter system. While the literature often tends to present 
these matrices as a somewhat heuristic construction, work in preparation of Broue, Malle, and 
Michel shows that when is a primitive (complex) reflection group, M is uniquely determined 
under a suitable set of natural axioms [33] . 

As a final preliminary, we recall that an element $ G Uch(l^) is special if the largest powers 
of X dividing the polynomials FakeDeg(<I>) G N[x] and Deg(<^) G M[x] are equal. Every special 
$ G Uch(T^) belongs to the subset Irr(VF). Such characters play an important role in the theory 
of unipotent characters for finite reductive groups and have been classified by Lusztig [23l Chapter 
4]. We mention also that Proposition 13.11 below provides an apparently novel description of the 
special irreducible characters of classical Coxeter systems. 

The Fourier transform M acts on functions / : Uch(VF) — t- C by matrix multiplication, since 
we may view / as a vector whose entries are indexed by Uch(VF). The following theorem, which is 
our main result, extends [20l Theorem 1] and [28J §6.4] to define a function on Uch(l^) which we 
might naturally view as the Frobenius-Schur indicator. The proof of this result appears at the end 
of Section HI 

Theorem 1.2. Suppose {W,S) is a finite, irreducible Coxeter system with associated Fourier 
transform matrix M. There then exists a unique function e : Uch(VF) — )■ M such that 

(1) e($) G {-1,0,1} for all $ G Uch(iy). 

(2) e(<I>) = if and only if the Frobenius eigenvalue of <I> G Uch(VF) is not real. 

(3) (Me)(<I)) is the multiphcity of ^ in xw for each special <I> G Irr(VF) C \Jch{W). 
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For this function e, it in fact holds that 

Xw= (Me)WV'- (1.2) 

t/'eIrr{Vl^) 

Furthermore, if (W, S) is crystallographic, then e coincides with the Frobenius-Schur indicator on 
Uch(Ty); i.e., e{^) is 1 or according to whether Eig($) is real or non-real for <I> G Uch(Ty). 

Conditions (1) and (2) are basic properties one would desire of a prospective Frobenius-Schur 
indicator. In particular we can restate (2) as the requirement that e($) = if and only if <I> is not 
fixed by an operator on Uch(l^) which we reasonably view as complex conjugation (see Proposition 
12.41 below). The simplicity of condition (3) and the surprising fact that it implies (jl.2p make the 
function defined in the theorem an attractive extension of the Frobenius-Schur indicator to the 
non-crystallographic case. Admittedly, this is not evidence that e is the "right" extension, only 
that it is a suitable and interesting one. 

This Frobenius-Schur indicator gives rise to two quaternionic "unipotent characters" in type 
H4. We record this phenomenon and some other properties of e which become evident in the proof 
of Theorem 11.21 in the following proposition. 

Proposition 1.3. Let e : Uch(VF) — {—1,0, 1} be the function defined in Theorem 11.21 

(a) e(^>) = 1 for all ^> G Irr(VF). If {W, S) is classical, then e($) = 1 for ah $ e Uch(VF). 

(b) e(<&) = — 1 if and only if {W, S) is of type H4 and <I> is either of two elements of Uch(II^) with 

Deg(<I>) = + higher powers of x. 

(c) (Me)(5') is a nonnegative integer for all ^ G Uch(l^). If {W,S) is classical, then (Me)($) is 
nonzero if and only if <I> is special. More generally, (Me)(<I>) is nonzero only if $ G Irr(H^) or 
if $ a single element of Uch(l^) \ Irr(VF) in each of the types F4, Eg, and H/i^. 

Remark. A lengthy computation shows that in type H4, there is no symmetric matrix M satisfying 
(PI) and (P2) for which there exists a function e : Uch(VF) — )• {0, 1} such that p.2p holds. Thus, in 
this sense even a different choice of Fourier transform matrix in type H4 still leads to quaternionic 
unipotent characters. 

It is an interesting open problem to describe how much of the preceding theory extends from 
Coxeter systems to complex reflection groups. For some but not all such groups, there are analogous 
notions of "unipotent characters" and Fourier transforms (see ^lOj), for which one expects some 
meaningful generalization of Theorem 11.21 to hold. 

Independent of any connection to unipotent characters, the representation qw is itself an in- 
teresting thing to study (e.g., see |lj which analyzes qw in type An). When W is classical of type 
An, BCn, or Dn, it is a natural problem to describe the irreducible decomposition of gw in terms 
of the familiar sets of partitions a, bipartitions {a, (3), and unordered bipartitions {a, (3} indexing 
Irr(VF). One can find such a description in Kottwitz's paper [20j— in particular, Kottwitz shows 
that the set of irreducible constituents of xw are precisely the special characters of W when W 
is classical (note also part (c) of Proposition II. 3p . We include the following purely combinatorial 
statement of these results for completeness. (See Section [3.11 below for a precise explanation of how 
we label the elements of Irr(VF); our conventions should be self-explanatory to those familiar with 
the subject.) 
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Theorem 1.4 (See Kottwitz [20]). For classical Coxeter systems {W,S), the character xw decom- 
poses as follows: 

• Type An. 

XW = ^x", 

a 

where the sum is over all partitions a of n + 1. 

• Type BCn. 

where the sum is over all bipartitions (a, f3) of n such that /3j < ctj + 1 and a[ < f3'^ + l for all 
positive integers i, and where d{a,/3) is the number of cells in the Young diagram of a H /3 
which occur simultaneously at the end of a column of a and at the end of a row of /3. 

• Type Dn. 

where the first two sums are over all partitions a of ^ (of which there are none if n is odd), 
where the third sum is over all bipartitions {a, (3) of n such that a C /3 and j3\a contains no 
2x2 squares, and where e(a, (3) is the number of connected components of /3 \ a. 

Remark. Let W be the Coxeter group of type BCn and W' C W the index two subgroup of 
type Dn- It is interesting to note that while we may view Invol(M/^') as a ^>vF-iiivariant subspace of 
Invol(Ty), the restriction of qw on this subspace to W is not equivalent to the representation qw'- 

We will actually derive this theorem from a more detailed statement, given as Theorem 13.21 
below, describing the irreducible decomposition of the subrepresentations Qw,a for each involution 
a S W. These results are well-known in type A^, see (HUH] in addition to [20j. 

As a corollary to the preceding two theorems we note the following result, which is interesting 
to compare with |361 Theorem 1]. Here, a Gelfand model is a representation of a finite group which 
is the multiplicity-free sum of all of the group's irreducible representations. 

Theorem 1.5. If {W,S) is a finite, irreducible Coxeter system then the representation qw is a 
Gelfand model if and only if {W, S) is of type An, H^, or him) with m odd. 

Our final results concern a conjecture of Kottwitz connecting the decomposition of xw to the 
left cells of W. (See Section [5] for the definition of the left cells and left cell representations of W.) 

Conjecture 1.6 (See Kottwitz [20]). Let L be a left cell in W and let xr denote the character of the 
corresponding left cell representation. Let a £ W he an involution and write S for its conjugacy 
class in W. Then {xw,(T,Xr) = |S n r|, where (•,•) denotes the standard L^-inner product on 
functions W ^ C. 
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Kottwitz [20] observed that in type the conjecture is true, following from the known descrip- 
tion of the left cells of W in terms of the RSK-correspondence. Casselman [T2] meanwhile verified 
the conjecture in types F4 and Eq by a computer calculation. In Section [5] below, we show ourselves 
that the conjecture holds in all of the non-crystallographic cases H^, H4, and l2(m). We note here 
that a weakened version of the conjecture follows immediately from Theorem 11.21 and recent work 
of Geek [ig. 

Theorem 1.7. Let T be a left cell in W and let xr denote the character of the corresponding left 
cell representation. Then the inner product {xw: Xr) is equal to the number of involutions in F. 

Proof. Proposition 11.31 and property (P3) of the Fourier transform matrix imply that (xw^Xr) = 
X]i/'eirr(VK)(V'5 Xr), which is the cardinality of the set {a G F : cj^ = 1} by Theorem 1.1]. □ 

We organize this article as follows. In Section [21 we note several preliminaries concerning 
Coxeter systems, the representation gw, and the set Uch(14/^). We reproduce the calculations 
needed to prove Theorems 11.41 and 11.51 in Section [31 In Section [H we describe in detail the Fourier 
transform matrices attached to Uch(iy) and derive from this information the proof of Theorem 
11.21 Finally, in Section [5] we prove Kottwitz's conjecture in the non-crystallographic types H3, H^, 
and l2{m). This last section explicitly describes the left cells in these types, and includes a proof 
of property (P3) for the Fourier transform matrix of Uch(l^). 
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2 Preliminaries 

Throughout, we adopt the following notational conventions: N is the set of nonnegative integers, 
P is the set of positive integers, and [n\ is the set of the first n positive integers, with [0] = 0. 



2.1 Representing W in Invol(I4^) 

Let (VF, 5) be a finite Coxeter system with length function £ : — )• N, and define the vector space 
Invol(M^) as in the introduction. Here, we briefly confirm that the map qw given in Definition 11.11 
indeed extends to a representation of W . The easiest way of deriving this from known results is to 
prove a slightly more general fact, which goes as follows. 

For each constant A; G Q, let Qw,k ■ S — )• GL(Invol(Ty)) be the map given by the formula 



0w.k[s)a'u 



if SW = ws and s ^ DesR,(u)), 

if SW = ws and s G DesR(it;), for s G 5 and w £ W with w'^ 
if SW / ws, 



Here we have written DesR(u') = {s £ S : i{ws) < £{w)} for the right descent set of w G W, 
which coincides with the left descent set when w'^ = 1. In this notation, the map gyy from the 
introduction is precisely gwo- 
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Define T-iqi as the Hecke algebra with parameter corresponding to {W,S): for us, tliis is the 
unital associative Q[(7]-algebra with basis {Tyj : w £ W} and multiplication given by the rules 

jTyjT^> = T^^f, for w,w' £W with £{ww') = £{w) + £{w'), 

\(T, + l)(T,-g2) =0, fors€S. 

In [28l [29] , Lusztig and Vogan show that the multiplication defined for s G 5 and w £ W hy 
J- 

qa^j + {q + l)asw, if sw = ws and s ^ DesR(t(;), 

_ \ {q'^ - q - + iq'^ - q)asw, if sw = ws and s G DesR(t(;), 

s — \ 

Usws, if sw 7^ ws and s ^ DesR(t(;), 

(g^ — l)a^ + q^asws, if stf 7^ tfs and s G DesR(t(;). 

makes the Q[g]-module Invol(14/^) i^q Q[g] into an ?^q2-module. (It is interesting to compare this 
with the Hecke algebra module in [1], which in type An is isomorphic to the one here once we 
replace (7 in [1] with q^.) Specializing q to 1 shows that qw,2 is a well-defined ly-representation, 
and from this we deduce the following stronger statement. 

Proposition 2.1. Let {W,S) be a finite Coxeter system. 

(1) The map Qw,k extends to a representation of W in Invol(VF) for any /c G Q. 

(2) The representations Qw,k fmd Qw,k' ai'e isomorphic for all /c, k' G Q. 

Proof. Let n{w) denote the dimension of the — 1-eigenspace oi w m. its geometric representa- 
tion. Clearly n{sws) = n{w) for all w € W and s G S, and it is a straightforward exercise, from 
well-known properties of the geometric representation, to show that n{sw) = n{w) -|- 1 if tt;^ = 1 
and sw = ws and s ^ DesR(t(;). 

List the involutions wi,W2, ■ ■ ■ jWn G in an order such that i < j implies n{wi) < n{wj). 
With respect to this ordered basis, the matrices of Qw,kis) for s G S are all block lower triangular 
and their diagonal blocks are independent of k. Given this observation, it follows that Qw,k extends 
to a VF-representation for all A; G Q if and only if it does so for a single nonzero value of k. As this 
occurs for k = 2, part (1) follows. The same observation shows that the trace of any product of 
the matrices gw,k{s) for s G 5 has no dependence on k, so gw,k and gw,k' have the same character, 
which proves (2). □ 

2.2 References for the construction of Uch(VF) 

In this section we provide references for the construction of Uch(VF) for each finite, irreducible 
Coxter system {W, S), and also for the associated data FakeDeg, Deg, and Eig. 

Even before discussing Uch(I^), we may give the general definition of the fake degree attached 
to a unipotent character. The fake degree of an irreducible character $ G Irr(VF) is the polynomial 
FakeDeg(<I>) G N[x] whose coefficients are the multiplicities of <I> in the graded components of the 
coinvariant algebra of W (see [11^ §2.4 and §11.1]). The set of irreducible characters Irr(l^) always 
forms a subset of Uch(W^), and we define the fake degrees of all <I> G Uch(l^) \ Irr(T^) to be zero. 
For $ G Irr(VF), Deg(<^) and Eig(<I>) are defined by 

Deg(<I>) = the generic degree of <I> (see [TTJ §10.11]) and Eig(<I>) = 1, 

while the polynomial FakeDeg(<I>) has the following well-known formula [IH Proposition 11.1.1]. 
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Proposition 2.2. For $ E Irr(Ty) we have 



FakeDeg($) = H 




i=l 



det(l — xw) 



where di, . . . ,de are the degrees of the basic polynomials invariants of W (see [HI §2.4]), and the 
determinant of 1 — xw is evaluated by identifying W with the image of its geometric representation. 

Explicit expressions for the right-hand side of this formula when (W, S) is classical appear in 
[111 §13.8]. We list the fake degrees in type l2{m) in Section [2.51 In the remaining exceptional and 
non-crystallographic types, one can readily compute the fake degrees using the proposition and the 
character table of W (see also O HI [6] ) . 

Now to describe Uch(VF) itself. When (VK, S) is crystallographic, Uch(H^) corresponds to the 
actual set of unipotent characters of an associated finite reductive group. Carter's book [11] contains 
an excellent exposition of this correspondence; Geek and Malle's paper [17l §2] also serves as a useful 
reference. We emphasize, however, that the construction of Uch(VF) is due originally to Lusztig 
[231 126j . as are the following facts. In brief, if G is a simple algebraic group defined over a finite 
field with q elements, with Frobenius map F : G ^ G and Weyl group {W,S), and if the finite 
group G^ = {g ^ G : F{g) = g} is split so that F acts trivially on W, then the following hold: 

• The number of unipotent characters of G^ , together with their Frobenius eigenvalues (as 
defined in [23', Chapter 11]), depends only on the isomorphism class of {W,S). 

• The degrees of the unipotent characters of G^ are given by the values at g of a set of generic 
degree polynomials, which also depends only on the isomorphism class of (W, S). 

A complete parametrization of Uch(VF) in these cases, together with a list of the associated degree 
polynomials, appears in [11, §13.8 and §13.9], while [23, Theorem 11.2] classifies in very simple way 
all of the Frobenius eigenvalues (see also Observation 12.71 below). From this description derives 
the following proposition, which summarizes some results of Lusztig [27] and shows that there is 
a natural definition of the Frobenius-Schur indicator on the formal set Uch(W) when (W, S) is a 
Weyl group. 

Proposition 2.3 (See Lusztig [27]). Let G be a simple algebraic group over an algebraically closed 
field of positive characteristic, with a Frobenius map F : G ^ G for which the finite group G^ is 
split. Then the irreducible unipotent characters of G^ all have Frobenius-Schur indicator 1 or 0, 
according to whether their Frobenius eigenvalues are real or non-real. 

Proof. Perhaps the simplest way to extract this result from [27] is to compare \23\ Theorem 11.2] 
with the description of e(<I>) (from |27] ) given in [28^ §6.4]. Alternatively, the result follows by 
comparing [23t Theorem 11.2] with \27\ Corollary 1.12] if G is classical, or with Table 1 and 
Proposition 5.6 in [14[ alongside the tables in [111 §13.9] or [23] if G is exceptional. □ 

When the Coxeter system (W, S) is non-crystallographic there is no corresponding reductive 
group, and the definition of Uch(VF) is instead based on heuristic arguments involving a list of pos- 
tulates considered plausible desiderata. Lusztig's paper [26] lists these postulates and constructs the 
set Uch(VF) with the associated degree polynomials when (W, S) is of any of the non-crystallographic 
types H^, H4, or l2{m). The corresponding Frobenius eigenvalues are given in |26| for H^,^ in [30] for 
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H4, and in |25j for l2{m); however, this information is not presented in the hterature as clearly as 
in the crystallographic case. Helpfully, one can access all the relevant data in types and i?4 (as 
well as in the exceptional types Eq, £'7, Eg, F4, and G2) from the UnipotentCharacters command 
in the computer algebra system CHEVIE [16]. To deal with the dihedral case, it is expedient to 
review an explicit construction of Uch(H^); we do this in Section [2.51 below. 

2.3 Two involutions A and j of Uch(W^) 

We now define two canonical involutions A and j of the set Uch(l^) which will be of importance 
in Sectional The first of these, A, arises from the observation that there is a well-defined notion 
of formal complex conjugation on Uch(VF). This fact derives easily from the descriptions we cited 
in the previous section, and we record it as the following proposition for future reference. 

Proposition 2.4. For each finite, irreducible Coxeter system {W,S), there exists a unique involu- 
tory permutation A of Uch(l^) with the following properties: 

(i) A preserves Deg(^>) and inverts Eig(<I>) for each $ G Uch(M^). 

(fi) A fixes $ G Uch(VF) if and only if Eig($) = ±1. 

The involution A is the identity permutation if and only if (W, S) is of classical type. Furthermore, 
in all types, A fixes all elements of Irr(14/') C Uch(H^) 

To define our second involution j, we recall that a polynomial f{x) is palindromic if there exists 
a nonnegative integer c E N such that f{x) = x'^ ■ f{x~^). In particular, the zero polynomial is 
palindromic. As noted by Opdam [35], the fake degree of nearly every $ S Uch(Ty) is palindromic 
and reflecting the coefficients of FakeDeg($) about a certain central power of x gives rise to a 
well-defined and meaningful involution of Uch(VF). (For Weyl groups W, this was first observed 
by Beynon and Lusztig [6j.) To explain what we mean, define a constant iV$ for each ^> G Uch(14/^) 
by the formula 

= $(iy Er ^(^) for $ G Irr(Ty) and iV$ = for $ € Uch(VF) \ Irr(Ty), 

where the sum on the left is over all refiections r €z W (i.e., those r £ W conjugate to elements 
of the generating set S). Also, let N denote the total number of refiections in W. The following 
proposition now summarizes several observations of Opdam [35, Page 448]. (In comparing this 
proposition to Opdam's paper, the reader should note the following misprint: the exponent of T 
on the right-hand side of [35; Eq. (2)] should be A'^^ — A^ and not A^ — A,-.) 

Proposition 2.5 (See Opdam [35j). For each finite, irreducible Coxeter system (W, S), there exists 
a unique involutory permutationion j of Uch(Pi^) with the following properties: 

(i) The fake degree of is • FakeDeg(<I>)(x"^) for each <I> G \]ch{W). 

(ii) j fixes $ G Uch(iy) if and only if FakeDeg(<^) is palindromic. 

The involution j is the identity permutation if and only if (W, S) is not of type E^, Eg, H3, or H4. 
Furthermore, in all types, j fixes all elements of Uch(iy) \ Irr(M^). 
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Remark. Under the assumption that TV is a Weyl group, this result first appeared as Proposition 
A in Beynon and Lusztig's paper [6J. There is also another interpretation of the involution j in 
terms of rationality properties of the corresponding characters of cyclotomic Hecke algebras; see 
[321 Theorem 6.5]. 

Even in types E^, Eg, H-^, and -ff4, the permutation j is very nearly the identity. In particular, 
adopting Carter's notation for the elements of Irr(Ty) (see our explanations in Sections 13.21 and 
13. 3p . we may describe the nontrivial actions of j on Uch(l^): 

• Type Ej. j exchanges 0512,11 with (/>5i2,i2- 

• Type Es. j exchanges (/'4096,ii with </>4096,i2 and (/>4096,26 with (?!)4096,27- 

• Type H^. j exchanges (j)4^3 with 04, 4. 

• Type Hi. j exchanges 0i6,3 with 0i6,6 and 0i6,i8 with 0i6,2i- 

One consequence of this description is that $ G Irr(Ty) is special if and only if j($(8)sgn) is special, 
a fact which Carter notes as fllj Corollary 11.3.10] when W is crystallographic. 

2.4 Families in Uch(14^) 

The set Uch(l^) possesses a distinguished decomposition into disjoint subsets called families. When 
(W, S) is crystallographic, these arise as the equivalence classes of a certain relation (see [HI 
Section 12.3]), but in general the families are defined heuristically (see [26] ). To specify the Fourier 
transform matrix of Uch(H^) it suffices to attach Fourier transform matrices to each of the families, 
and we therefore discuss some of their significant properties here. 

To begin, each family C Uch(Ty) contains a unique special element ^—special, we recall from 
the introduction, means that there exists an integer e G N and nonzero real numbers a, b such that 

FakeDeg(<^) = ax*^ + higher order terms and Deg(<I>) = bx'^ + higher order terms. 

Since FakeDeg($) = if ^> ^ Irr(PF), each special ^ necessarily belongs to the subset of irreducible 
characters Irr(VF) C Uch(l^). If <I> is the unique special element of a family and e is defined as 
above, then is also the largest power of x dividing Deg(^') for all other ^ £ J^. Furthermore, 
if is the largest power of x dividing FakeDeg(^') for some ^ £ J- Ci Irr(VF), then e < e' unless 
$ = ^. 

From this discussion we see that each family in Uch(IF) has a nonempty intersection with 
Irr(VF). Thus the division of Uch(VF) into families induces a similar partition of Irr(VF) into families. 
The family decomposition of Irr(II^), in contrast to that of Uch(VF), has a simple combinatorial 
definition in terms of the two-sided cell representations of W which applies in all types. Namely, a 
family in Irr(VF) consists of the characters appearing as constituents of two-sided cells having the 
same special constituent; see [Til §12.4] for a detailed explanation. 

Let A and j be the involutions of Uch(T^) defined in Propositions 12.41 and 12.51 above. Every 
family in Uch(IF) is preserved by both of these permutations, and we make the following definition 
concerning the action of j on a family. 

Definition 2.6. An element of Uch(II^) is exceptional if it is not fixed by the involution j, or 
equivalently if its fake degree is not palindromic. (All such characters are listed at the end of 
Section [2.3i ) A family in Uch(VF) is exceptional if any of its elements are exceptional. 
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Remark. This notion of exceptionality originates in Beynon and Lusztig's paper [6] and has since 
become a standard definition, appearing in various places [201 [35]. All exceptional families have size 
4, and they only occur in types -Ey, Eg, H^, and H^. In types Ej and H3, Uch(VK) contains exactly 
one exceptional family, while in types Eg and H4^, \]ch{W) contains two exceptional families. 

As Lusztig first observed |23j, a single construction provides an extremely convenient way of 
parametrizing nearly every family in Uch(VF). Given a finite group F, let Cr{x) = {g G T : gxg~^ = 
x} denote the centralizer of an element x £T, and define ^(F) as the set of equivalence classes of 
pairs (x,(t) for x G F and a G Irr(Cr(x)), with respect to the relation 

{x,a)^{gxg-\a'^) for 5 G F,. (2.1) 

Here denotes the character of Cr{gxg~^) with the formula z 1— a{g^^zg). Apart from one 
family in type H4 and one family in type /2(?n.), each family C Uch(H^) is naturally in bijection 
with a set ^{T) for some finite group F, given either by a product of 2-element cyclic groups or a 
symmetric group. In particular, each has size 

1, 8, 21, 39, 74, 2^'', k"^, or + k + 2 (where A; is a positive integer) , 

and the families of size 74, A;^, and /c^ + A; + 2 occur only in types H^, l2{2k + 1), and l2{2k + 2), 
respectively. 

If {W, S) has one of the classical types An, BCn, or then the families in Uch(VF) each 
correspond to sets ^ ((Z/2Z)'') for various integers k > 0. If {W,S) has type Eq, Ej, E^, F4, or 
G2, then the families in Uch(I^) each correspond to ^{Sk) for some k e {1,2,3,4,5}. A detailed 
list of the families J- C Uch(H^) when (VK, S) is crystallographic, alongside the corresponding 
bijections F -H- ^{T), appears in [11, §13.8 and §13.9]. With respect to these correspondences, the 
following observation is a consequence of [231, Theorem 11.2]: 

Observation 2.7. Suppose (W,S) crystallographic and J- C Uch(iy) is a family parametrized 
as in [Ilk §13.8 and §13.9] by the set ^(F) for a finite group F. Let <I> G J-" be the element 
corresponding to (x,(t) G ^(F). 

(a) If F is not exceptional then Eig(^>) = ^||y and A($) is the unique element of T corresponding 
to the equivalence class (x,a) G ^(F). 

(b) If T is exceptional then F = 6*2 and Eig($) = 1 if x = 1 and Eig($) = (j{x) ■ i otherwise. 
In this case, A fixes the elements of J- with Frobenius eigenvalue 1, and exchanges the two 
elements with Frobenius eigenvalue iti. 

Furthermore, in either case $ is special if and only if {x,a) = (1, 1), where 1 denotes the principal 
character of F. 

Carter's book [llj does not similarly address the non-crystallographic case; however, in these 
types, only a small number of families exist. A description of these families appears in Lusztig's 
papers papers [251 126] . which we may summarize as follows: 

• Type H-^. There are 7 families: 4 of size 1 and 3 of size 4. 

• Type ^^4. There are 13 families: 6 of size 1, 6 of size 4, and 1 of size 74. 
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• Type l2{m). There are 3 families: they are described by ()2.2p below. 

One can find an exact parametrization of the families in types and Hi^ (and also for the crystal- 
lographic exceptional types) in the computer algebra system CHEVIE [16]. We will discuss these 
families at greater length in Section [4.31 

We close this subsection by considering the related combinatorial problem of counting the 
elements of ^(F). If F is abelian, then ^(F) = F x Irr(F) and we have |^(F)| = |Fp. If F is a 
symmetric group, then we have the following less trivial result. 

Proposition 2.8. Let a„ = |^(5„)| and write cr(n) for the sum of the positive divisors of a 
positive integer n. The ordinary generating function of the sequence {an}^=i then satisfies 



Hence, in the language of [5], {a,i}^^ is the Euler transform of the sequence {a{n)}'^^]^. 

Remark. The sequence {an]n=i = (1> 4, 8, 21, 39, 92, 170, 360, . . . ) appears as [34, A061256]. 

Proof. If C\ denotes the centralizer in Sn of a permutation with cycle type A, then a„ is equal to 
sum over all partitions A of n of the number of conjugacy classes of C\. It is not difficult to check 
that if the partition A has parts of size r, then Cx = H-rM ^('"' "^r)' where G(r, n) = {'L/r'L)lSn 
denotes the wreath product of a cyclic group of order r with Sn- 

The conjugacy classes of G(r, n) are indexed by partitions of n whose parts are labeled by 
numbers i G [r\. Given a sequence (/ir)r>i of such labeled partitions indexing a conjugacy class in 
C\ = Y\y.yiG{r,mr), modify the r^^ partition ji^ by multiplying its parts by r and replacing all 
labels i by ordered pairs (r, z). Concatenating the parts of these modified partitions yields a map 
from the set of conjugacy classes of C\ to the set Vn of partitions of n whose parts of size k are 
labeled by pairs {d,i) where d is a divisor of k and i G [d]. Conversely, any u G Vn determines a 
sequence of labeled partitions [^ir)r>i which indexes a conjugacy class of some C\: namely, take 
/ir to be the labeled partition given by dividing by r all parts of u which are labeled by pairs of 
the form (r, i). These operations determine a bijection from Vn to the disjoint union of the sets of 
conjugacy classes of C\ over all partitions A of n, and thus a„ = \Vn\- 

We can view Vn as the set of partitions of n whose parts of size k can have a{k) different types. 
The cardinality of this set is what is counted by the coefficient of in the right-hand generating 
function above, essentially by construction (see [5]), which completes the proof. □ 

2.5 Unipotent characters in type l2{'m) 

It seems difficult to find a single reference which provides all of the data our setup requires to 
construct Uch(l^) in type l2{m). We therefore briefly include the relevant details here, with the 
papers [25^ [26l [3lj serving as our primary references. 

Fix m > 3 and let {W, S) be of type him). The set Uch(VF) then consists of the objects 

^>(0j) for integers j with < j < 

for integers i,j with 0<i<j<i + j<m, 
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together with the additional objects 

{1, sgn, if m is odd 

1, sgn, ^'i^Q my <1>|q m)) if TO is even. 

Observe that Uch(Ty) has cardinahty A:^ + 2 if m = 2/c + 1 is odd or /c^ — /c + 4 if m = 2A; is even. 

Let ^ = exp ^ ^^^^ ^ be the standard primitive m^^ root of unity. The fake degrees and degrees 
for Uch(M/^) are then defined by 



x^ + x""-^, ifi = 0, 
0, otherwise, 



FakeDeg 

FakeDeg ($(o,™) ) = FakeDeg (^(o,-) ) = 



and 



Deg {^i^,J)) 



{x-S,^)(x-^"^-'){x-^:'){x-^"^-3)^ 

Deg (cl>'(, = Deg [l>'l, ,.^) = ^ • x • Uo<k<fi^ " " r''^), 

with FakeDeg (1) = Deg(l) = 1 and FakeDeg (sgn) = Deg (sgn) = x™. Note that in the right-hand 
expression for Deg ,,■)), the denominator of the last factor always divides its numerator, and so 
the degree does belong to ]R[x]. The Frobenius eigenvalues have the formula 



Eig(«>) 



^ if $ = for some 
1, otherwise. 



We note that in this type, the permutation A of Uch(Ty) defined in Proposition l2.4l fixes all elements 
except those of the form ^(ij) with i > 0, on which it acts by A : ^(ij) ^ ^{i,m,-j)- 
Finally, Uch(H^) always has exactly three families, given by the sets 

{1}, {sgn}, and Uch(H^) \ {1, sgn}. (2.2) 

The special elements of Uch(VF) are then 1, sgn, and *5(o,i)- 



3 Decomposing xw for arbitrary Coxeter systems 

Kottwitz and Casselman's papers [201 E] give the decomposition of xw when is a Weyl group. 
Here we explain the derivation of one combinatorial reformulation of Kottwitz 's results for classical 
Coxeter systems, then describe the irreducible decomposition in the remaining non-crystallographic 
types. 
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3.1 The combinatorial statement of Kottwitz's results for classical types 

We devote this section to the proof of Theorem 11.41 in the introduction. To begin, we recall the 
following explicit constructions of the Coxeter systems of classical type: 

• Type An- W is the symmetric group 5„+i of permutations of [n + 1] and S = {si, . . . , Sn} 
where Sj is the simple transposition + 1) € Sn+i for i £ [n]. 

• Type BCn- W is the group of n x n generalized permutation matrices with entries in 
{—1,0,1}, a group we occasionally call Wn- The set S = {si, . . . , s„_i, t„} consists of the 
permutations Sj G Sn (viewed as matrices) together with the matrix t„ = diag(l, 1, . . . , 1, — 1). 

• Type Dn- W is the subgroup of matrices in the Coxeter group of type BCn with an even 
number of entries equal to —1. The generating set S = {si, . . . , s„_i, t^} consists of the 
permutations Si G Sn (viewed as matrices) together with the matrix = tnSn-itn- 

To refer to the irreducible representations of these groups we employ the following conven- 
tions. First, a partition of an integer n is a weakly decreasing sequence of positive integers 
a = («!, • • • 5 Ofc) which sum to n. For convenience we define to be for all i G P ex- 
ceeding k and write |a| for the sum X^jgptti- The Young diagram of a partition a is the subset 
of P X P given by {(i,j) : 1 < J < Oi}) which we draw in "English notation" as in the following 
example: 

a = (4, 2, 1) has Young diagram 



We often identify a partition a with its Young diagram; for example, we write a <Z (3 when the 
Young diagram of a is contained in that of /3, and write a H /3 for the partition whose i^^ part is 
min{aj,/3j}. We denote the transpose of a by a'; recall that = |{j G P : ctj < i}|, and that 
consequently the Young diagram of a' is the transpose of the diagram of a. 

All Coxeter groups {W, S) have two linear characters given by the trivial character 1 : w i— )• 1 
and the sign character sgn : w i— )• (— 1)^^"'). The remaining irreducible characters of the the classical 
Coxeter groups may be described as follows: 

• Type An- For each partition a of n -|- 1, there exists a unique irreducible character 

of = Sn+i whose restrictions to the subg roups Sq^^ X S(y^2 ^ * * * ci-rici Sq^' X >S*q,/ X • • • 
respectively contain the trivial character and sign character as constituents. The characters 
for partitions a of n -|- 1 are distinct and exhaust Irr(l^). 

• Type BCn- If a is a partition then the irreducible character of S^^^ extends in two ways 
to characters x+ and X- of ^\a\ formulas 

xT(5)=x"(bl) and ^-(s) = A(5)x"(bl), for g G W\^\, 

where \g\ denotes the permutation matrix formed by replacing every entry of g with its 
absolute value, and A(g) is the product of the nonzero entries of g. Given an ordered pair of 
partitions (a, /3) with |a| + \/3\ = n (i.e., a bipartition of n), define X^'^'^) as the character of 
W = Wn induced from the character x+ X- of the subgroup W\a\ x W^|/3|- The characters 
for bipartitions (a,/3) of n are distinct and irreducible, and exhaust Irr(VF). 
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Type Dn- Define an unordered bipartiton of n to be a set of two distinct partitions {a,/3} 
with |a| + = n. If {a, (3} is an unordered bipartition of n then the irreducible characters 
and x^^'"^ of Wn have the same, irreducible restriction to W, which we denote by x^"''^^ ■ 
If n is even and a is a partition of n/2 then the restriction of ^^"''^^ to W has two irreducible 
constituents, which we denote in no particular order by x^°^'^ ^-iid x^"^^'"^- The characters 
X'f^'^J' for unordered partitions {a, /3} of n together with x^"^'^ X^"^''^ for partitions a of 
n/2 exhaust the distinct elements of Irr(H^). 



Lusztig \23\ Chapter 4] describes an alternate indexing set for the irreducible characters of the 
classical Coxeter groups, based on equivalence classes of certain two-line arrays. This notation in 
useful in providing simple conditions for two characters in Irr(l^) to belong to the same family 
in Uch(VF) or for a character to be special, and in fact these conditions seem to be the way most 
authors define such things. Nevertheless, we can describe at least the special characters in Irr(T^) 
easily enough without introducing any such further notation. This description does not appear to 
be well-known and may be interesting from the viewpoint of combinatorics. (For the definition of 
the families in Ivv{W), see [III §13.2].) 

Proposition 3.1. The special irreducible characters of the finite Coxeter systems {W, S) of classical 
type are described as follows: 

• Type An. is special for all partitions a of n -|- 1. 

• Type BCn. X^"''^^ is special if and only if < Oi + 1 and a- < PI + I for ah i G P. 

• Type Dn. x^"'^^ is special if and only if one of the partitions a, /3 properly contains the 
other and the corresponding skew diagram contains no 2 x 2 squares. Both x^"'^'^ X^"'^''^ 
are special for all partitions a of n/2 when n is even. 

Remarks. Before giving its proof, let us explain the proposition pictorially. 

(1) The condition for x^"'^^ to be special in type BCn corresponds to the following picture: 



If the Young diagram of a is the set of gray cells, then special if and only if /3 is 

formed from a by adding (a subset of) white cells and/or removing (a subset of) gray cells 
marked by "'s. 

(2) The condition for x^'^'^^ to be special in type D„ may be alternately stated as the requirement 
that a C /3 and /3i+i < ai + 1 for all i G P (or that the same condition hold with a and /? 
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reversed). The relevant picture here is the following: 



If the Young diagram of a is the set of gray cells and a C /3, then special if and 

only if /3 is formed by adding (a nonempty subset of) white cells to a. Note that our diagram 
truncates the set of white cells shown in the first row and first column. 

Proof of Proposition \3.1[ That every irreducible character in type An is special is well-known [ll^ 
Proposition 11.4.1]. The condition in type -BC„ for a character ;^("'^) to be special |1H Propositions 
11.4.2 and 11.4.3] is usually stated as the requirement that Ai < < A2 < ^ " " " ^ ^ ^m+i 
where 

Xi = am+i-{i-i) + and fii = Prn-ii-i) + - 1), (3.1) 

and where m is any integer large enough so that m + 1 and m are at least the number of parts of 
a and /3, respectively. In particular, one can take m = n since |a| + |/3| = n. It is simply a matter 
of rewriting this statement to deduce that special if and only if Oj+i < /3i < + 1 for all 

i G P. In turn, it is straightforward to check that a^+i < Pi for all i G P if and only \i a[ < I3'- + 1 
for all i e P. 

Similarly, in type Dn the character special if and only if Ai < //i < • • • < Am < /Um or 

A*! < Ai < • • • < /im < Am where 

A, = am-(i-i) + («-!) and /ij = /3m_(i_i) + (3.2) 

and m is any sufficiently large integer fll. Proposition 11.4.4]. This is equivalent to the requirement 
that < /3j and < + 1 for all i G P (or the same condition with a and /3 reversed), and it 
is straightforward to check that this holds if and only if a C /3 and /3 \ a contains no 2 x 2 squares. 
Finally, Carter [IH Proposition 11.4.4] notes that x^""'^'^ and x^"'^''^ are special in type D2n for all 
partitions a of n. □ 

The involutions of W are the permutations whose cycles all have length one or two in type A^,', 
and the matrices which are symmetric in type BCn or Dn- For each nonnegative integer m, let Um 
denote the permutation (l,m + l)(2,m + 2) • • • (m, 2m) or (when appropriate) the corresponding 
2m X 2m permutation matrix. The permutations a"m for < m < [^^^J represent the distinct 
conjugacy classes of involutions in the Coxeter group of type An, while in type BCn, the matrices 




for integers k,i,m > with 2m + k + i = n 
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provide the desired set of representative involutions. In the type D„, the matrices (Tk,i,m with 
2m + k + £ = n and i even each belong to a distinct conjugacy class of involutions. If n is odd 
then any involution is conjugate to one of these elements; if n is even, then there is one additional 
conjugacy class of involutions represented by the product of cro^o,m/2 and the diagonal matrix 
diag(l, . . . , 1, —1, —1), an element we denote (Jq q ^^g- 

As the main result of this section we now describe how the characters xw,a for involutions 
a & W decompose when {W, S) is of classical type. Kottwitz derives equivalent forms of these 
decompositions in his paper [20], and our primary contribution is to collect his findings in one 
concise, combinatorial statement. 

Theorem 3.2 (See Kottwitz [20j). For a finite Coxeter system {W,S) of classical type, the char- 
acters XW,a for involutions a £ W decompose as follows: 

• Type An- Let m be a nonnegative integer with 2m < n + 1. Then 

XW,(7m = y^X") 

a 

where the sum is over all partitions a of n + 1 with exactly n + 1 — 2m odd columns. 

• Type BCn- Let A;, m > be integers with 2m + k + i = n. Then 



a n /3| — m 



where the sum is over bipartitions (a, /3) with \a\ = k+m and |/3| = i+m such that /3j < a^ + l 
and a^< I3'- + 1 for alH G P, and where 

d{a, (3) is the number of pairs (i, j) G P x P for which i = a'j and j = /3i. 

Type Let A:, ^, m > be integers with 2m + k + £ = n and £ even and k + £ ^ 0. Then 

" (f;;)V/K/3)-^r ' 

where the sum is over all bipartitions (a,/3) with \a\ = m and |/3| = k + £ + m such that 
a C f3 and /3 \ a contains no 2 x 2 squares, and where 

e{a, (3) is the number of connected components of the skew diagram /3 \ a, 
f{a,f3) is the number of positive integers i with Oi 7^ /3j. 

In addition, if n is even then one can label the characters X^"^'^; X^"^'^ ^ Irr(PF) such that 

a a 

where both sums are over all partitions a of n/2. 



17 



Remark. Theorem 11.41 in the introduction follows by summing each of these decompositions over 
the corresponding representative set of involutions, noting that we evaluate the binomial coefficient 
(^) to be zero if A; > n or < 0. 

Concerning the decomposition in type Z?„, we recall that the connected components of a skew 
diagram are the equivalences classes of cells under the relation of (left-right or up-down) adjacency. 
For example, if /3 = (7, 5, 2, 2) and a = (5, 4, 1) then the skew diagram /3 \ a, shown as the white 
cells in the picture ________ 




has three connected components. 

Proof. The decomposition of xw,ij in type An is well-known; see, for example, pTJ Corollary A.l] or 
[T9l Lemma 2] or [20, §3.1]. 

In type BCn, given a bipartition (a, /?) of n, Kottwitz [201 §3.2] proves that the multiplicity of 
in XW,LUk I m ^^'^^ unless is special and \a\ = k + m and = i + m, in which case 

the multiphcity is {\anp\-m)^ where 

• d is the number of integers in the sequence //i, //2) ■ ■ ■ j Mn which do not occur in the sequence 
Ai, A2, . . . , A„+i, where Aj = Q!„+i_(i_i) + (i - 1) and m = + («-!) as in (I3T|. 

Given Proposition 13. H we have only to show that when special, d is equal to the number 

d{a, /?) defined in the theorem. As noted in the proof of Proposition 13. H if special then 

Ai < /Ui < • • • < A„ < /i„ < An+i, and so d is equal to the number of i € [n] with Xi < fii < Aj+i, 
which by definition is the number of i G P with a^+i < Pi < on. This last formulation is precisely 
the number of pairs (i, j) G P x P for which i = a'j and j = Pi, as desired. 

Now assume (W, S) is of type Kottwitz notes the given decompositions of Xvy,CToon/2 
Xw,a'gg 1^ when n is even in |201 §3.3]. On the other hand, for an unordered bipartition {«,/?} 

of n, Kottwitz [20I §3.3] proves that the multiplicity of in Xw,bJk i m (assuming I is even 

and + £ 7^ 0) is zero unless 

^{a,/3} is 

special and {|a|, |/3|} = {fc -|- £ + m,m}, in which case the 

multiplicity is (^_^^_^J, where 

• e is the number of integers in the sequence ^i, ^2, ■ ■ ■ , fJ-n which do not occur in the sequence 
Ai, A2, . . . , Xn, where we define Aj = a„_(j_i) + {i — 1) and /Xj = f3n-{i-i) + (^ ~ 1) as in (|3.2p . 

• /o is X^jgpmax{/?j+i,ai} if a C /? or X^-gp max{aj+i, /3j} if /? C a. 

In light of Proposition 13.11 it suffices to show that if special then is equal to the 

coefficient (j^^^"^^!^) defined in the theorem. 

We may assume a C /3 and (3 \a contains no 2 x 2 squares. Then Ai < ;Ui < A2 < /i2 ^ ■ ■ ■ < 
Xn ^ A*n) and e counts the number of i G [n] with Xi < fii < Aj_|-i, where by convention A„+i = 00. 
Equivalently, e is equal to the number of i G P with on < Pi < Oj-i, where we define uq = 00. 
Consulting the picture in the remarks following Proposition 13.11 one finds that the upper right 
most cells of the connected components of /3 \ a are precisely those of the form {i. Pi) for i G P with 
Oii < Pi < Ui-i. Hence e = e{a,P). 
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Continuing, it follows from our assumptions that /o = |a| +go = m + go, where go is the number 
of i G P for which ctj + 1 = /3j+i. The number of connected components of /3 \ a is equal to /(a, /3) 
(the number of i G P with < (3i) minus g^, since gQ counts the nonempty rows in /3 \ a which are 
connected to the rows above. Thus = (^^J = (e+gV^) = (/&?-^)' desired. □ 

3.2 The decomposition of xw in type H-^ 

Let (W, S") be the Coxeter system of type H^, so that W is isomorphic to the direct product of 5*2 
and the alternating subgroup of Ss, and S = {a, c} consists of three elements, which we label 
according to the Dynkin diagram 

a c 

(so that a,b,c satisfy the relations (ab)^ = (ac)^ = (bc)^ = = b'^ = = 1). There are then four 
conjugacy classes of involutions in W, represented by the elements 1, a, oc, and (abc)^, and W has 
10 irreducible characters. We refer to these, mirroring Carter's notation in |11] . by writing (l)d,e for 
the irreducible character of W satisfying 

0d,e(l) = d and FakeDeg(</)(i_e) = (nonzero constant) • + higher order terms. 

This convention manages to identify each element of Irr(VF) uniquely. 

Decomposing gw and its subrepresentations is a routine calculation which we have carried out 
in the computer algebra system Magma [8]. From this computation, we obtain the following: 

Proposition 3.3. If (W, S) is of type H3 then the characters xw,a- decompose as follows: 

(1) Xw,i = 4>i,o- 

(2) Xiy,(a6c)5 = 4>1,15- 

(3) XW,a = 4>3,1 + 4>3,3 + 04,3 + (p5,5- 

(4) XW,ac = (1)3,6 + 4>3,8 + 04,4 + 05,2- 

Consequently, xw = Z]i/'eirr{iy) ^ ^^^^ gw is a Gelfand model. 

3.3 The decomposition of xw in type H4 

Let {W,S) now be the Coxeter system of type H4, so that W is a finite group of order 14400 
generated by four elements S = {a, 6, c, d}, which we label according to the Dynkin diagram 

a — - — b c d . 

There are five conjugacy classes of involutions in W, represented by 1, a, ac, {abcf' , and {abcdY^ , 
and Irr(VF) has 34 elements. We refer to the irreducible characters of W exactly as in type H^, 
with one exception: in type if4, the notation (j)d,e fails to distinguish the two irreducible characters 
of W of degree 30, and we consequently denote these characters by 03o,io,i2 and 03o,io,i4: here, 
030,10,/ indicates the irreducible character with 

03O,io,/(l) = 30 and FakeDeg(03o,io,/) = x^^ + + higher order terms. 

Again, decomposing xw and xw,a- is a routine computation, which we have carried out in Magma. 
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Proposition 3.4. If {W, S) is of type H4 then the characters xw,o- decompose as follows: 

(1) Xw,i = 4>i,o- 

(2) XW,(a6cd)l5 = 01,60- 

(3) XW,a = 4>4.,1 + </'4,7 + 016,3 + 036,5- 

(4) XW,{abc)5 = 04,31 + 04,37 + 016,21 + 036,15- 

XW,ac = 09,2 + 09,6 + 09,22 + 09,26 + 016,6 + 016,18 + 025,4 + 025,16 

(5) 

+ 2024,6 + 2024,12 + 2018,10 + 2030,10,12 + 2030,10,14 + 2040,8- 

3.4 The decomposition of xw in type l2{rn) 

Fix m > 3 and let (W, S) be the Coxeter system of type l2(jn), so that W is the dihedral group of 
order 2m, generated by the two elements S = {r, s} subject to the relations = = (rs)"^ = 1. 
The involutions in W are the elements 1 and {rsyr for < j < m — 1, along with (rs)*"/^ if m is 
even. One checks that 



DesR(l) = 0, DesR ((rs)"/^^ = {r, s}, DesR {{rsYr) = < 



{r}, if 2 j + 1 < m, 
{r,s}, if2j + l = m, (3.3) 
if 2j + 1 > m. 



Write wq for the longest element of W, given by (rs)~2~r if m is odd or (rs)™'/^ if m is even. The 
group W has either two or four conjugacy classes of involutions, represented by 1 and r if m is odd 
and by 1, wq, r, and s if m is even. 

The irreducible characters of W are given as follows. There are two linear characters when m 
is odd, given by 0i.o = 1 and 0i.m = sgn, and four linear characters when m is even, given by 

01,0 = 1, 0i,m = Sgn, 0; ,„/2 : r^s'' ^ (-l)^ and 0'/^^/2 ■ ^'^ ^ ("l)^ • 

There are in addition [^^^^J distinct irreducible characters of degree two given by the functions 

02fc: (rsyr i-)- 0, r ■ . , ^ , 

; (1 „ -7 / N integers k with < K < — . 

{rsy H> 2cos(27rjA;/m), ^ 2 

These constructions exhaust all elements of Irr(VF). We have labeled the characters of W following 
our convention in type and H4: the first index of <j3d,e indicates the character's degree while 
the second index is the largest power of x dividing the character's fake degree. In the notation of 
Section 12.51 we have 

I' _ ^/ J." 



)l,0 = 1, (pl^m = Sgn, <Pl,m/2 = ^(0,f )' 01,m/2 = ^(0,f ) ' 02,fc = ^(O,fe)- 



Table [U describes the irreducible decomposition of the characters xw,a- and xw in type hirn). 
The rows in this table correspond to individual irreducible characters of W, while the columns list 
the multiplicity of each row in the characters xw,a- 

Three distinct patterns arise according to the residue class of m modulo 4. The proof of the given 
decompositions is a simple exercise using Frobenius reciprocity and the fact that xwa is induced 
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from a linear character A of the centrahzer Cw{o') of a in W. The character A is determined 
by formula gwig)0'(7 = '^(s')«cr for g G (7^/(0"), which may be explicitly evaluated using (13. 3p . In 
addition, one checks that if m is odd and a is one of the representative involutions 1 or r then 
Cw{o') is W or = S2, and that if m is even and a is 1, wq, r, or s then Cwic^) is W, W, 

{l,wo,r,wor} = S2 x S2, or {l,wo,s,wos} = S2 x 82- Evaluating the inner product of A with the 
elements of Irr(VF) restricted to these subgroups provides the desired multiplicities of xw,a- 
Summarizing Table [H we have the following proposition. 

Proposition 3.5. Suppose {W,S) is of type l2{m) with m > 3. 

m — 1 

(1) If m is odd then xw = 4^1,0 + 0i,m + Efcii '?^2,fc = T.^&rr{W) V'- 

m-2 

(2) If m EE 2 (mod 4) then xw = 4>lfi + (t)l,m + (t>l,m/2 + <t>'i^rn/2 + Yjk=l 2(/'2,2fc-l- 

m 

(3) If m = (mod 4) then xvy = 0i,o + <Pi,m + I]fc=i 2</)2,2fc-i- 



4 Fourier transform matrices and the proof of Theorem 11.21 

Here we describe the Fourier transform matrices associated to Uch(H^) for each finite, irreducible 
Coxeter system (Wi 5), and derive from this setup the proof of Theorem ll.2[ In the crystallographic 
case, the relevant definitions are well-established and due originally to Lusztig [23]. We must take 
more care to describe the associated matrices for the non-crystallographic types, as the literature 
[Hi [25l [301 [31] presenting this heuristic theory is not nearly as cohesive or extensive. 

We first describe how to attach to each family in Uch(I4^) (see Section [2. 4p a Fourier transform 
matrix M. The Fourier transform matrix of Uch(VF) is subsequently constructed as the direct sum 
of such matrices M over all families. In Section 14.41 we describe some notable properties of this 
Fourier transform and discuss in what sense these properties indicate the choices of matrices M to 
be "canonical." 



4.1 Fourier transform matrices in crystallographic types 

As described in Section [2^ when (VF, S) is crystallographic, every family in Uch(Ty) is parametrized 
by a set ^(F) for some finite group F. (Recall from (12. ip the definition of this set.) Lusztig |23j 
defines the Fourier transform associated to a family indexed by ^(F) as the matrix 



Mr = (4.1) 



whose entries are the numbers 



{(., a), {y, r)} ^r^^^ E - {gyg-') r [g-'x-'g) 

for {x,a),{y,T) G ^(F). Carter helpfully provides an explicit description of the sets ^(F) and 
the accompanying matrices Mr in the cases when T = Sn and n < 4 (as well a partial matrix in 
the case F = 55) [TT, §13.6]; see also the overview in [11, §12.3]. We review the frequently occuring 
case F = 5*2 in the following example. 
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Example 4.1. If s denotes the nontrivial element of then ^(5*2) = {(1, 1), (1, sgn), (s, 1), (s, sgn)}, 
and with respect to the order in which we just listed ^(S'2), the corresponding matrix is 





/ 1 


1 


1 


1\ 


1 


1 


1 


-1 


-1 


2 


1 


-1 


1 


-1 




V 1 


-1 


-1 


1 / 



4.2 Fourier transform matrices in type /2(m) 

The description of the Fourier transform in this case comes from Lusztig's paper |25j . Assume 
{W, S) is of type I2{itl) for an integer m > 3 and recall the explicit construction of Uch(VF) given 
in Section [2.5[ As noted there, Uch(VF) has only three families, two of which have size one: {1} 
and {sgn}. The Fourier transform of both 1-element families is the 1x1 identity matrix. 

Let T = Uch(Vl^) \ {l,sgn} denote the remaining family, and define X = X' U X", where X' 
and X" are the disjoint sets given by 

X' = |Pairs of integers (i, j) with i + j < m and either 0<i<j<morO = i<j<Y|, 

f{(0,f)',(0,fn, if mis even, 
1 0, otherwise. 



X" 



As is clear from our notation in Section [2.51 X naturally parametrizes J-. Write ^ = exp y ^'l^ 

for the standard m**^ root unity. The Fourier transform of T, as defined by Lusztig [25|, is then 
the matrix 

whose entries are the numbers {x, x'} given by 
and if m is even and € X' , by 

{(i, j), (0, f )'} = {(i, j), (0, f n = {(0, f y, = {(0, f r, = 
{(0, f )', (0, f )'} = {(0, f r, (0, f n = i^^^ + i, 
{(0, f )', (0, f n = {(0, f r, (0, f )'} = '-^^ - \. 

We have labeled this matrix as a mnemonic for "dihedral Fourier transform." 
Example 4.2. It is helpful to review some examples of this construction. 

(i) If m = 3 then X = {(0, 1)} and D3 is the 1x1 identity matrix. 

(ii) If m = 4 then X = {(0, 1), (0, 2)', (0, 2)", (1, 2)} and with respect to an appropriate ordering of 
indices we have D4 = • In a similar way, one finds that Dg = M53 . These equalities are 
consistent with the fact that the Coxeter systems of types /2(4) and 12(6) are isomorphic to 
those of types BC2 and G2, whose nontrivial families of unipotent characters are parametrized 
by J^{S2) and J^{S^). 
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(iii) If m = 5 then X = {{0, 1), (0, 2), (1, 2), (1, 3)} and with respect to the order in which we just 
hsted X, one computes 



V5 



/a a 1 1 \ 

A A -1 -1 

1-1 A -A 

\ 1 -1 -A A / 



where A = ^ ~^ and A = — . 



This is precisely the matrix hsted in [[9, Eq. (7.3)] and in [251 §3-10]. 



4.3 Fourier transform matrices in types and 

The definition of the Fourier transform in the remaining non-crystallographic cases comes from 
the papers O EDI EI]- In type H^, Uch(VF) has four 1-element families given by {</>i,o}, {</>!, 15}) 
{4'5,2}, and {(^5,5}. In type //4, Uch(W^) has six 1-element families given by {01,0}, {0i,6o}i {^'25,4}, 
{4'25,i6}, {4>36,5}, and {i;^36,i5}- These families are all subsets of Irr(VF) and we have labeled their 
(necessarily special) elements according to our conventions in Sections 13.21 and 13.31 To all such 
1-element families, the associated Fourier transform matrix is the 1x1 identity matrix. 

In type (respectively, H4), Uch(Ty) has three (respectively, six) 4-element families. One such 
family is exceptional in type (see Definition 12. 6p . while two are exceptional in type H4. The 
following observations concerning these families are derived from the parametrizations of Uch(VF) 
provided by the UnipotentCharacters command in CHEVIE |16j . First, in both types the excep- 
tional families always consist of four elements 



(1,1)' 



(l,sgn)) 



(s,sgn) 



which can be indexed by the set ^(82), such that 

• ^(i,i)'^(i,sgn) e Irr(VF), with $(1^1) special and Deg (^(1,1)) = Deg ($(i,sgn))- 

• ^(s,i),^(s,sgn) i Irr(W^), with Deg ($(,,1)) = Deg ($(.,sgn)) and | ^^J^'^^^ 

Remark. The computer algebra system CHEVIE stores a large amount of data associated to 
Uch(l^), which can be accessed by combining the commands Display and UnipotentCharacters. 
The parametrization by ^{82) just given, however, is not included in CHEVIE, though the listed 
properties uniquely determine which index (x, cj) G ^{82) goes to which $ S for any exceptional 
4-element family T. 

The characters *^(i,sgn) ^ Irr(VF) may be respectively either (ji^^^, (j)A,4 in type i^s or i?i)i6,3, 

016,6 or (/'i6,i8! '/'i6,2i in type i/4. There is no established notation for the formal elements *I'(s,i)! 
sgn) in each family, however. 

The non-exceptional 4-element families in types and i/4, on the other hand, always consist 
of four elements 



^(0,1)) ^(0,2)5 ^(1,2)> 



'(1,3) 



which can be indexed by the set X in Section [4.21 with m = 5, such that if ^ = exp , ^ 
fifth root of unity, then 



2nV^ 



IS a 
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^(0,1)' ^(0,2) e Irr(T^), with $(o,i) special and Deg (^'(o,i)) / Deg (^'(o,2))- 
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• ^(1,2), <f (1,3) i Irr(W), with Deg ($(1,2)) = Deg ($(1.3)) and J ^(^''^ \ 

I^Eig ($(1,3)) =i^. 

This parametrization by X, though uniquely determined for each non-exceptional family, is again 
not actually listed in CHEVIE. The characters <^(o.i)) '^'(0,2) ^ Irr(M^) may be either (\>-i^\^ i?!>3,3 or 
(/)3,6, 03,8 in type H-i or any of the pairs (/)4,i, (/)4,7 or 04,3i, 04,37 or ^9,2, 09,6 or ^9,22, 09,26 in type 
i?4. There is again no established notation for the formal elements $(1,2)) ^(1,3) in each family. 

The Fourier transforms of these families are now defined thus: if C Uch(VF) is a 4-element 
family in type -ffs or ^^4, then its Fourier transform matrix is 

^ I M52, if T is exceptional (see Example 14. ip . ^ 
1 D5, if T is not exceptional (see Example I4.2( iii)). 

Note that these assignments make sense because we have indicated how each 4-element family is 
indexed by the same set as the corresponding matrix. 

Remark. In our definition of the Fourier transform matrix for the non-exceptional 4-element 
families, we follow the convention of [9l §7]. The Fourier transform of the exceptional 4-element 
families in types -ffs and Hi^ seems less well-established in the literature. The matrix assigned to 
these families here is chosen to be identical to the Fourier transform matrix of the other exceptional 
families in types E-j and . We will say more about the "correctness" of this choice in the remarks 
following Theorem 14.61 below. 

These conventions attach a Fourier transform matrix to all but one remaining family in type H/^. 
In this type, Uch(VF) has a single family T of size 74; the intersection of this family with Irr(H^) 
has size 16 and its unique special element is the character 024,6 £ Irr(VF). The Fourier transform 
of this family is constructed by Malle as the matrix S in his paper [30]. To do calculations with 
this matrix, one needs to be able to access it in some computer format, and the algebra package 
CHEVIE fortunately provides this capability. In detail, one can obtain the 74 x 74 Fourier transform 
matrix of T by the following sequence of CHEVIE commands in GAP: 

W := CoxeterGroup("H",4); 

Uch := UnipotentCharacters(W); 

F := Uch.f amilies[l3]; 

M := F.f ourierMat * MatPerm(F.perm, 74); 

The odd-looking multiplication by MatPerm(F.perm, 74) in the last line has to do with the indexing 
conventions of the f ourierMat field in CHEVIE. The code given here produces a matrix M whose 
rows and columns have the same indices as Uch, and which is identical to the one in |30j . 



4.4 Observations and consequences 

From the preceding three subsections, we know of a Fourier transform matrix attached to each fam- 
ily T in Uch(VF) for each finite, irreducible Coxeter system (VF, S). The Fourier transform matrix 
of Uch(VF), we reiterate, is the direct sum of these matrices over all families J-. By construction. 
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this Uch(VF)-indexed matrix satisfies property (P2) in the introduction; i.e., it is block diagonal 
with respect to the decomposition of Uch(I^) into families. The following theorem explains pre- 
cisely how this matrix also satisfies property (PI) in the introduction. This statement should be 
attributed to Lusztig and Malle via a combination of results appearing in the papers p3 | [25 t [30 1 [31] : 
see the proof of [T71 Theorem 6.9] for a detailed bibliography. 

Theorem 4.3. Let (W, S) be a finite, irreducible Coxeter system with associated Fourier trans- 
form matrix M, and write j for the (permutation matrix of the) involution of Uch(T^) defined in 
Proposition 12.51 Then the composition M o j transforms for the vector of fake degrees of Uch(VF) 
to the vector of (generic) degrees. 

Remark. This result seems to be well-known, but is often stated imprecisely. For example. Carter 
asserts, without any mention of j, that the Fourier transform matrix of Uch(Ty) transforms the 
vector of fake degees to the vector of actual degrees whenever {W,S) is a Weyl group [HI §13.6]. 
This statement, at least as we interpret it, is not strictly true in types and Eg. In particular, 
one can check that the Fourier transform matrix M that Carter attaches to the three exceptional 
families in these types (see Section [2. 4p does not literally transform the vector of fake degrees to the 
corresponding vector of (generic) degrees as listed in [111 §13.8]. But M does transform a nontrivial 
permutation of the fake degrees to the (generic) degrees. 

The preceding theorem gives one reason to consider the particular Fourier matrices assigned 
to Uch(VF) as somehow "canonical," and our next result provides another. To state this, we first 
must recall Lusztig's definition of a fusion datum [25] . 

Definition 4.4. Let X be a finite set with a distinguished element xq, and suppose 

• A is the matrix of an involutory permutation of X with xq as a fixed point; 

• M is a real symmetric matrix indexed by X; 

• F is a diagonal matrix indexed by X whose diagonal entries are complex roots of unity. 
The tuple {X, xq, A, M, F) is a fusion datum if the following axioms hold: 

(Commutability). M = AMA and F"^ = AFA. 
(Positivity) . Mx^xo > for all x G X and Fxq^xq = 1- 
(Modularity). = (FAM)^ = 1. 

(Integrality). E^„ex ^--"ffj^--^-'- G N for ah x,y,ze X. 
Remarks. A few comments are helpful in unpacking this not altogether transparent construction. 

(a) In [25j, Lusztig actually presents a more general definition of a fusion datum which involves 
two involutions [J, b of X in place of A. This definition reduces to ours when [j = b = A. 

(b) The modularity axiom is so-named as it requires that the matrices A, F, M determine a 
unitary representation of the modular group PSL2(Z). 
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(c) The integrality axiom allows one to define an algebra structure with nonnegative integer 
structure coefficients on the complex vector space generated by X. More specifically, the 
axiom asserts that one can give this vector space the structure of a based ring in the sense of 
[24] : see the discussion in [I7l §7]. 

Before proceeding we note the following short lemma, which identifies a common type of fusion 
datum based on the set ^{T) attached to a finite group T. 

Lemma 4.5. Let F be a finite group equal to (Z/2Z)" or Sn for some n > 0, and suppose 

• X = ^{T) as defined by and xq = (1, 1) € X; 

• A : X —7- X is the involution defined by A : {x,a) i— t- 

• M = Mr as in (gj]); 

• F = diag(ta;)^ex where = ^ for (x,cr) G .£{V). 

Then (X, xq, A, M, F) is a fusion datum in the sense of Definition 14.41 

Proof. This is equivalent to |251 Proposition 1.6], provided we show that {x,a) ~ {x~^,a) for each 
(x,ct) G ^(r) when T is either (Z/2Z)" or Sn- This is immediate in the first case, so assume 
r = Sn is a symmetric group. 

Let A be a partition of n and let x E 5'„ be a permutation with cycle type A. In the proof 
of Proposition 12. 8^ we noted that Cs„ [x) is isomorphic to the direct product of wreath products 
ElrM C!{r,mr), where rrir is the number of parts of A with size r. A less explicit version of this 
statement goes as follows: the cycles of x generate an abelian group A = Yl^yi Z/AjZ, and Cs„{x) 
is isomorphic to a semidirect product of the form x Sm^ x . . .) tK A. It is easy to see that one 
can choose a permutation g G Sn which commutes with the left factor of this semidirect product 
and which has gxg~^ = x"^ and in fact g~^yg = y^^ for all y £ A. Noting these properties, it 
follows from the standard construction of the irreducible characters of a semidirect product with 
an abelian normal subgroup (see \2T, Exercise XVIII. 7]) that = a for any a G Irr(C5„(x)). We 
conclude that = {gxg~^ = {x~^,a) for all {x,a) G ^{Sn), as required. □ 

With these preliminaries in tow, we may now state the following noteworthy result due to Geek, 
Lusztig, and Malle [T71 |25l |30], showing how each family in Uch(H^), combined with its attached 
Fourier transform matrix, possesses naturally the structure of a fusion datum. This observation 
elaborates property (P4) of the Fourier transform matrix noted in the introduction, and is essentially 
a special case of ^17( Theorem 6.9] (although as stated, that result drops the positivity axiom of a 
fusion datum and uses somewhat different terminology.) 

Theorem 4.6. Let (W, S) be a finite, irreducible Coxeter system. Suppose 

• J-" is a family in Uch(VK) and <I>o G is its unique special element; 

• A is the restriction to J- of the involution of Uch(iy) given in Proposition 12. 4t 

• M is the Fourier transform matrix of as defined in the Sections 14.11 14.21 14. 3t 

• F = diag (Eig($)) $gjr is the diagonal matrix of Frobenius eigenvalues of $ G J^. 
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Then [T, ^q, A, M, F) is a fusion datum. 



Remark. For almost all families in Uch(Ty), our assignment of Fourier transform matrix follows 
exactly the convention established in the computer algebra system CHEVIE [16]. For the six 
exceptional families in types E^j, Eg, H^, and H4, however, our assigned matrix differs slightly 
from the one stored in CHEVIE— although, these matrices are the same after a permutation of 
rows and/or columns. In justification of our assignments, we can say the following: if one assumes 
Uch(VF) given, then for each exceptional family, there is a unique matrix M satisfying both Theorem 
14.31 and Theorem 14.61 So at least in this sense our choice of M is canonical. 

We include a brief proof of the theorem for completeness. 

Proof. In the case that {W, S) is crystallographic and is a non-exceptional family of Uch(I4^), the 
theorem follows by combining Observation 12.71 with Lemma 14.51 If is one of the six exceptional 
families in types E^, Eg, H^, or H4, then since is A-invariant, the theorem is equivalent to the 
claim that {X, xq, A, M, F) is a fusion datum for X = {1, 2, 3, 4} and xq = 1 and 



/ 1 \ 

10 

1 

V 1 y 



M = - 
2 



/ 1 
1 

1 

V 1 



1 \ 

-1 

-1 
1 / 



diag(l,l,z. 



The proof of this is a straightforward computer calculation. All families not covered by these cases 
occur when (W, S) is non-crystallographic, and are either singletons; non-exceptional 4-element 
families in type or ^^4; the nontrivial family in type /2(m); or the 74-element family in type H4. 
In the first case the theorem holds trivially; in the second and third case, the tuple {J-, ^q, A, M, F) 
coincides with dihedral fusion datum given in [251 §3]; and in the last case, (J^, <I>o, A, M, F) is by 
construction the fusion datum which Malle describes in [30]. □ 



We conclude this section finally by proving our main theorem from the introduction. 

Proof of Theorem \1.2l If (W, S) is classical, then the function e : Uch(Ty) — > R which is identically 
1 satisfies (l)-(3) in Theorem 1 1 . 2 1 and also (II. 2p by j20^ Theorem 1]. The uniqueness of this function 
e follows by Theorem 14.61 as a consequence of the positivity axiom of a fusion datum, since any 
function e' : Uch(VF) — ^ M satisfying (1) and (2) must have e'(<I>) < e{^) for ah $ G Uch(I^). The 
final statement in the theorem holds because the Frobenius eigenvalues of Uch(iy) all real if {W, S) 
is classical by Observation 12.71 (noting that the group T in Observation O is in this case). 

Assume {W,S) is of type /2(m), let M denote the Fourier transform matrix of Uch(VK), and 



be the function with e(<I>) = if <I> = *I*(ij) for some 0<i<j<i+j<m 



let e : Uch(PF) - 

such that j 7^ y and with e(<I>) = 1 for all other $ G Uch(I1/^). From the discussion in Section [2.51 
this function satisfies conditions (1) and (2) in the theorem. To show that it also satisfies (3) and 

exp ( ) and 0<i<j<i + j<m, 



(|1.2p . it suffices by Proposition 13.51 to check that if ^ 
then when m is odd, we have 



-jk 



0<fc<f 



0<fc<f 



if i = 0, 
otherwise, 
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and when m is even, we have 



Me ($(,,)) = E [{{^,J), (0, A;)} + {k, ^)}) + (0, f )'} + {(z,i), (0, ^)"} 

0<k<f 



0<fc<f 



2, if i = and j is odd, 
0, otherwise. 



and also, for x £ {(0, f )', (0, f )"}, 



2 ^ ' 2 
def 



M6($.)= E {^,(0,fc)} + {x,(fc,^)} +{x,(0,^)'} + {x,(0,^r} 



o<fc<^ 



i_(_i)"i/2 j 1, if m = 2 (mod 4), 
~ 2 ~ jo, ifm = 0(mod4). 

(Note that we need to check that Me(l) = Me(sgn) = 1 as well, but this is obvious.) Proving each 
of these three identities is straightforward. Thus the funciton e satisfies conditions (l)-(3) in the 
theorem, and we deduce that it is the only such function by the positivity axiom of a fusion datum, 
exactly as in the classical case. 

Suppose finally that (W, S) has one of the remaining exceptional types. If {W, S) is not of type 
H4, then one can compute directly using the tables in [T2] and Proposition 13.31 that the function 
e : Uch(VF) — )• M which is 1 or according to whether Eig(<l>) is real or non-real satisfies (l)-(3) as 
well as (II. 2p . That e is the unique function satisfying (l)-(3) then follows as in the previous cases 
by Theorem 14.61 and the positivity axiom of a fusion datum. Similarly, if (W, S) has type H4, then 
one can calculate using Proposition 13.41 that (l)-(3) and (jl.2p hold for the function e : Uch(VF) — )• M 
which is on all <I> G Uch(M^) with Eig(<I>) ^ M, — 1 on the two elements of Uch(Ty) whose 
degrees have the form + higher powers of x, and 1 on all other elements of Uch(VF). Here, 
the uniqueness of e does not follow immediately from Theorem 14.61 because the values of e are 
not all positive, but it can still be easily checked. In this case, the positivity axiom of a fusion 
datum implies that nearly all of the functions e' : Uch(H^) — t- R satisfying (1) and (2) must have 
Me'(<I>) < Me($) for one of the special characters ^ £ Irr(iy). The remaining list of functions on 
Uch(W^) which could possibly satisfy (l)-(3) is quite small, and a short calculation confirms that 
the given function e is indeed the only one with these properties. □ 



5 Left cells and Kottwitz's conjecture 

In this final section we investigate how the preceding material connects to the left cells of a Coxeter 
group, and prove some partial results related to Conjecture 11.61 in the introduction. 

Let (W, S) be a finite Coxeter system with length function i : W ^ N. For each pair of elements 
y,w G W, we write 

Py,^(x) gN[x] 

for the associated Kazhdan-Lusztig polynomial, as defined in [71 Chapter 5] or |1H §12.5] or \18\ 
§7.9], among other places. Recall that Pw,w{x) = 1 and that Py^w{x) = unless y < w, where < 
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denotes the Bruhat partial order on W. We now make the following sequence of definitions for 
y,w gW: 

• Write y ^ w li y < w and Py^w{^) has degree ^ {^{w) — £{y) — 1). 

• Write y <l w if there exist elements xq^ xi, . . . , Xk £ W such that y = xq and w = Xk and for 
each i G [k], these two conditions hold: (1) either ~< Xi or Xj ~< Xj_i, and (2) there exists 
s £ S such that i{sxi-i) < £{xi-i) and £{sxi) > i{xi). 

• Write y ~l u; if y <i w and u; <l y. 

The left cells of are the equivalence classes of the relation Let Vr = Q-span{c^ : u; G T} be 
a vector space with a basis indexed by a left cell T in W, and define a map qy ■ S ^ GL(Vr) by 



£»r(s)c„ 



Zw, if ^(si^) < ^(it;), 

+ n{y, w)cy, if i{sw) > i{w), for s G 5 and wGT, 

yer, e{sy)<i{y) 



where fi{y,w) is the coefficient of x^^^^^^ ^^^^ in Py^^ix) (which is zero unless y -< w). This 
extends to a representation of W, called the left cell representation of F, whose character we denote 
by Xr- 

The following properties of left cells and left cell representations are useful to recall (see [71 
Chapter 6]). First, the left cell representations decompose the regular representation of W and so 
the sum over left cells xr is equal to J2ji>£irr{w) ^(l)V'- Second, the singleton set {1} is always 
a left cell, and its character is the trivial character 1. Finally, if wq denotes the longest element of 
W and r is a left cell, then wqT and Twq are left cells and XwqT = XVwo = Xr • sgn. 

We now state a few less well-known facts about the characters xr- 



Theorem 5.1. If F, F' are left cells in a finite Coxeter group W, then {xr,Xr') = |F n F' ^|. 

Proof. When is a Weyl group, this is \23\ Proposition 12.15]. Alvis notes how to extend Lusztig's 
proof to type [2, Proposition 3.4], and his argument remains valid in types and l2{'m) (noting 
the explicit description of the left cells for these groups given in the following sections.) □ 

Corollary 5.2. If F is a left cell in a finite Coxeter group W, then xr is multiplicity- free if and 
only if every w £ T Ci F~^ is an involution. 

Proof. The character xr is multiplicity- free if and only if the inequality (xr , Xr) > X^i/)Girr(VK) (xr , V') 
is an equality. The left side is |Fn F~^| by the previous theorem, while the right side is the number 
of involutions in F by [15\ Theorem 1.1]. □ 

The next theorem shows the Fourier transform matrix of Uch(H^) indeed satisfies property (P3) 
in the introduction. 

Theorem 5.3. Let (W, S) be a finite, irreducible Coxeter system with Fourier transform matrix 
M, as in Section HI Fix a left cell F of W, and let v be the vector indexed by Uch(PF) whose entries 
are the irreducible multiplicities of xrj extended by zeros on Uch(VF) \ Irr(Ty). Then = v. 

We prove this here, even though to do so we must appeal to a few results not yet given. 
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Proof. When is a Weyl group, this is \23\ Theorem 12.2]. In types and H4, the theorem 
follows from a computer calculation using the description of the characters xr given in Sections 
5.11 and 15.21 below. In type l2{m) it follows by a short argument using the content of Sections 

4.21 and 15.31 that the theorem is equivalent to the following identities: if ^ = exp ^ ^^^^^ ^ and 
< i < j < i + j < m, then when m is odd 



0<fc<2t 



1, if i = 0, 
0, otherwise. 



and when m is even 



_^ ^ j_ (^^-ik _^ ^ik _ ^jk _ ^-jk 

0<k<f 



1, ifi = 0, 
0, otherwise. 



l_(_l)m/2 ^ _ 1 

2m / J m 2 



0<fc<f 

Proving each of these is straightforward arithmetic. □ 

The last object of this section is to prove the following theorem. We will accomplish this in a 
case-by-case fashion, by examining the left cell representations in each non-crystallographic type. 

Theorem 5.4. Coniecture 11.61 holds if {W, S) has type H3, H/^, or l2{m). 

From this and the results of \12\ 120] . we see that Kottwitz's conjecture holds for all finite 
irreducible Coxeter groups except possibly those of type BCn, Dn, E^, and Eg- (Furthermore, 
computer calculations indicate that the conjecture holds in types BCn and D„ for at least n < 6.) 

5.1 Left cells of the Coxeter group of type H-^ 

Let {W, S) be the Coxeter system of type -^3. The group W decomposes into 22 distinct left cells 
which we may describe as follows. (This description does not seem to appear anywhere in the 
literature, though one can easily compute the left cells in type directly, using for example Fokko 
du Cloux's program Coxeter [13], which is what was used to derive the following statements.) 
Label the generators S = {a, 6, c} as in Section [3^2] and let wq = (abc)^ denote the longest element 
of W. Following Alvis [2], we define 

Rj = {w £ W : DesR(u)) = J}, for each subset J C S, (5.1) 

and let X* = {wqw : w £ X} for any subset X C W. We now define 12 subsets Ij, Jj, Ki C W as 
follows: 

-^1 = R{b,c} n R{a}aba, Ji = R{a,c} n R{c}cbab, Ki = R^a} - {h U J3), 

I2 = ha, J2 = Jib, K2 = R{b} - {h U Ji U J2 U J4), 

h = hb = II, J3 = J2a, K^ = R(^c}-h- 
Ii = ha = Il, J4 = J3b, 

J5 = J4C, 
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The reader should compute that = 8 and |Jj| = 5 and \Ki\ = 6. In addition, let L = {1} 
so that L* = {wq}. We now have this computational proposition (which is closely related to the 
calculations summarized in [22| §5]): 

Proposition 5.5. The left cells of the Coxeter system [W, S) of type H^, are the 22 disjoint subsets 
li, Ji, J* , Ki, K* , L, L* , and the characters of the associated cell representations are respectively 

(04,3 + (/"m); 4>5,2] 4>5,5; i4>3,i + 4>3,3); {4>3,6 + 4>3,8); 4>i,o; </'i,i5- 

Proof. Coxeter [13] outputs a description of the left cells and the associated cell representations in 
type in a few seconds. We have checked that this information matches what is asserted in the 
proposition using MAGMA [8j . □ 

Table [2] lists (alongside some helpful auxiliary data) the sizes of the intersections of the left cells 
in W with the group's four conjugacy classes of involutions, which we recall from Section 13.21 are 
represented by the elements 1, a, ac, and (abc)^. Each row of the table corresponds to a left cell, 
while each of the last four columns corresponds to a conjugacy class. Comparing Proposition 13.31 
with Table [2] yields a proof of Conjecture 11.61 in type by inspection. 



5.2 Left cells of the Coxeter group of type H4 

The Coxeter group of type H4 decomposes into a disjoint union of 206 left cells. Alvis classifies 
these in [2], and assigns each of them one of the labels Ai, Bi, B*, Ci, C* , Di, D*, Ei, E*, Fi, F* , 
Gi, Gi- We refer to [2] for the precise definition of these sets, noting the following correction: 

Remark. On page 162 of the published version of Alvis's paper [2j, the left cell A12 is defined by 
the equation 

A12 = Aiod (INCORRECT) 

which should instead be 

A12 = And. (CORRECT) 

Apart from this quite minor (but inevitably confusing) detail, everything else in Alvis's paper seems 
to be completely accurate. 

Table [3] lists the sizes of the intersections of the left cells in W with the group's five conjugacy 
classes of involutions, which are represented as in Section [3.31 by the elements 1, a, ac, (abc)^, and 
{abcdy^. We have structured this table exactly like Table [21 and comparing it to Proposition 13.41 
similarly yields an immediate proof of Conjecture 11.61 in type H4. 

5.3 Left cells of the Coxeter groups of type him) 

Let {W, S) be the Coxeter system of type l2{m) for m > 3, with S = {r, s} and wq defined as 
in Section [3.41 The group W then decomposes into a disjoint union of four left cells, given by the 
singleton sets X = {1} and X* = {wq} together with 

Y = {s, rs, srs, . . . , (• • • srs) } = Rs^s} a-iid Y* = {r, sr, rsr, . . . , (• • • rsr) } = -Rj,,.}, 

m—1 factors m—1 factors 

with Rj defined by (jS.ip . As in type H-^, the following description of the left cell representations 
of W does not seem to appear anywhere in the literature, though it is easy to compute directly. 
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Proposition 5.6. The left cells of the Coxeter system {W,S) of type l2{fn) are the 4 disjoint 
subsets X, X* , y, y*, and the characters of the associated cell representations are respectively 

<Al,0, 4>l,m,, I]o<A;<f'^2,fc, I]o<A:<f<?^2,A;, if m is odd, 

4>i,m, </'i^„/2 + Eo<fc<f '?^2,fc, 'P'lm/2 + ^o<k<f^2,k, if m is even. 

Proof. The description of the left cells is noted, for example, in [18l §7.15]. We have xx = 4'i,o = 1 
and Xx* = 4>i,m = sgn automatically, and the remaining left cell characters are multiplicity-free by 
Corollary 15.21 Since the left cell representations decompose the regular representation and since 
XY* = XY ■ Sgn, we must have xy = XY* = ^"^M odd, and if m is even, we must have 

either xy = 4>'i^rn/2 + Sfc </'2,fc and xy* = <i>i^rn/2 + Z^fc '/^2,fc or the reverse assignments. One resolves 
the ambiguity when m is even by computing the values of the characters at r or s. □ 

Table m finally lists the sizes of the intersections of the four left cells in W with the group's two 
or four conjugacy classes of involutions (see Section [3741) . We have structured this table exactly like 
Tables [2] and [3l and comparing it to Table [1] completes the proof of Theorem 15.41 
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m odd 



Character 


Xw,i 


XW,r 


Xw 


01,0 


1 





1 


01,m 





1 


1 


02,fc (0 < A; < f ) 





1 


1 



m = 2 (mod 4) 



Character 


XH/,1 XH/,«)o XW,s 


Xw 


01,0 

01, m 

4>2,k {k odd) 

02, A; (fe even) 

0i,m/2 
01,n./2 


10 
1 
11 

1 
10 


1 
1 

2 


1 
1 


171 = (mod 4) 


Character 


XW,1 XW,wo XW,r XW,s 


xw 


01,0 

01, m 

02, *; {k odd) 
02,fe (fc even) 

0i,m/2 
0;'n./2 


10 
1 
11 





1 
1 

2 






Table 1: Irreducible multiplicities of xw,a and Xw in type l2ifn) 



Left cell 


Cell size 


Cell character 


1 


{abc)^ 


a 


ac 


/i (1 < i < 4) 


8 


04,3 + 04,4 








1 


1 


Jj (1 < i < 5) 


5 


05,2 











1 


J* (1 < i < 5) 


5 


05,5 








1 





ifj (1< z < 3) 


6 


03,1 + 03,3 








2 





if* (1 < ^ < 3) 


6 


03,6 + 03,8 











2 


L 


1 


01,0 


1 











L* 


1 


01,15 





1 









Table 2: Sizes of intersections of left cells and conjugacy classes of involutions in type 
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Left cell 


Cell size 


Cell character 


1 




a 


(a6c)^ 


ac 


Ai {l<i<8) 


326 


^24,6 + 024,12 + 024,7 














12 






+024,11 + 08,12 + 08,13 
















+018,10 + 030,10,12 + 030,10,14 
















+040,8 + 2048,9 












Ai{9<i< 18) 


392 


024,6 + 024,12 + 024,7 














14 






+024,11 + 010,12 + 016,11 
















+016,13 + 018,10 + 030,10,12 
















+030,10,14 + 2040,8 + 2048,9 
















'P24,6 + <P24,12 + 024,7 


U 


A 

u 


u 


A 

u 


1 R 
ID 






1 J. \ J. \ J. 

+024,11 + 06,12 + 06,20 
















+016,11 + 016,13 + 2030,10,12 
















+2030,10,14 + 2040,8 + 2048,9 












Bi{l<i< 36) 


36 


036,5 








1 








B* (1 < i < 36) 


36 


036,15 











1 





Ci{l<i< 25) 


36 


025,4 














1 


C; (1 < i < 25) 


36 


025,16 














1 


A (1 < ^ < 16) 


32 


016,3 + 016,6 








1 





1 


I?* (1 < i < 16) 


32 


016,18 + 016,21 











1 


1 


Ei {l<i< 9) 


18 


09,2 + 09,6 














2 


E* (1 < i < 9) 


18 


09,22 + 09,26 














2 


(1 < i < 4) 


8 


04,1 + 04,7 








2 








i^* (1 < i < 4) 


8 


04,31 + 04,37 











2 





Gl 


1 


01,0 


1 














G\ 


1 


01,60 





1 












Table 3: Sizes of intersections of left cells and conjugacy classes of involutions in type H4 
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m odd 



Left cell 


Cell 


size 


Cell character 


1 


r 




X 


1 




^1,1 


1 







X* 


1 









1 




Y 


m — 


1 


Y^o<k<f 4>2,k 





(m — 


l)/2 


Y* 


m — 


1 


X)o<fc<^ h,k 





(m — 


l)/2 



m = 2 (mod 4) 



Left cell 


Cell size 


Cell character 


1 Wq r S 


X 
X* 
Y 
Y* 


1 
1 

m — 1 
m — 1 


(t>l,m 

^'l,m/2 + T.O<k<f (t"i,k 
^l,m/2 + Eo<fe<f ^■^,k 


10 
10 

(m-2)/4 (m + 2)/4 
(m + 2)/4 (m-2)/4 


m = (mod 4) 


Left cell 


Cell size 


Cell character 


1 Wq r s 


X 

X* 

Y 

Y* 


1 
1 

m — 1 

m - 1 


</*l,m/2 + Eo<fc<f ^2,fc 


10 
10 

m/4 m/4 

m/-i. m/i 



Table 4: Sizes of intersections of left cells and conjugacy classes of involutions in type l2{m) 
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